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(54) Method and apparatus for managing a decoder buffer 



(57) The present invention relates to digital data 
transmission and more precisely a method and appara- 
tus for managing a decoder buffer. 

In data transmission applications, such as digital 
video broadcasting, large amounts of data are encoded 
and transmitted to decoders having a decoder buffer. 
Management of the decoder buffer is very important as 
transmittal of too little data can result in the decoder 
buffer underflowing (becoming empty), or if too much 



data is transmitted the decoder buffer could overflow. 
Both decoder buffer overflow and underflow result in 
loss of data and the decoding process is adversely 
affected. 

The present invention provides a method and appa- 
ratus for managing a decoder buffer such that decoder 
buffer overflow and underflow are minimised. 
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Description 

[0001 ] The present invention relates to digital data transmission and more precisely a method and apparatus for man- 
aging a decoder buffer. 

5 [0002] Standard resolution digital television uses a pixel resolution of 480 x 704 pixels with a frame rate of 30 frames 
per second. High definition television (HDTV) differs from standard television in having a much higher resolution. A 
number of differing HDTV standards have emerged, each differing in image resolution and/or frame rate. The table 
below illustrates some of the more common HDTV formats. 

10 

Scanning Format (lines x 
pixels x frames/s) 

15 1080x1920x30 

720x1280 x 60 
720x1280x30 
480 x 704 x 60 

20 

[0003] Due to the superior resolution of HDTV over standard television, dedicated encoders and receivers are needed 
to both encode and decode HDTV pictures. In our co-pending United Kingdom patent application, GB 9807203.6, a 
technique is described, however, which enables HDTV pictures to be encoded using a number of standard television 
25 encoders. This technique describes how a HDTV source is split up into a number of segments which are then sepa- 
rately encoded by standard television encoders. These separately encoded segments are then recombined and proc- 
essed to produce a HDTV compliant encoded signal. 

[0004] Standard television encoders are designed to encode video in accordance with the various video compression 
standards, such as the Motion Picture Expert Group standard (MPEG and MPEG-2). These standards specify the 
30 requirements of standard television decoders, including the size of the decoder buffers. Standard television encoders 
are therefore designed to produce an amount of encoded video information within the limits of a standard television 
decoder buffer. 

[0005] In a decoder buffer, data is received at either a constant or variable bit-rate. Video frames are removed from 
the decoder buffer at a constant rate as dictated by the frame rate. The data contained in each frame (or frame size) will 
35 differ depending on the way in which the frame was originally encoded. A decode time stamp (DTS) can be associated 
with each frame to indicate to the decoder when a frame should be removed from the decoder buffer. A frame can either 
be marked with a DTS or a DTS can be derived for a frame from the DTS of other frames. This helps ensure that the 
frame is decoded at the appropriate time. 

[0006] The size of frames differs widely due to the way in which frames are encoded. Some frames, known as I- 
40 Frames, are generally the largest since they contain information which allows a complete frame to be reconstructed by 
a decoder. Two other types of frame, known as B-Frames and P-Frames, are generally much smaller in size, since they 
are used in conjunction with a previously reconstructed frame to reconstruct a new frame. 

[0007] It is possible for an l-Frame to completely fill a decoder buffer. If the decoder buffer is already half full when 
such an l-Frame is sent, some of the l-Frame data will overflow the decoder buffer and be lost. This may cause the 

45 decoding process to breakdown. 

[0008] If data is received at the decoder buffer at a slower rate than which data is removed the decoder buffer may 
become empty (or underflow). If this were to happen, the decoder would try to remove a frame from the decoder buffer, 
but the entire frame would not be present. This may also cause the decoding process to breakdown. 
[0009] Further problems arise in the above-described situation where a number of standard television encoders are 

so used to produce a single HDTV signal. The decoder buffer requirements as defined for a HDTV decoder are such that 
the collective size of multiple standard television decoder buffers is greater than the defined decoder buffer size for a 
HDTV decoder. If each of the individual standard television encoders produces large amounts of data for the same 
frame this may cause the HDTV decoder buffer to overflow. 

[0010] Should a decoder buffer be allowed to overflow or underflow, this may cause the decoding process to break- 
55 down. Since decoders have no control over the data they receive, management of the decoder buffers must be per- 
formed at the transmission end of the system. . . _ . _ _ 
[0011] Tn a standard encoder, a decoder buffer model is usually maintained to monitor the actual performance of a 
real decoder buffer. If the decoder buffer model is in danger of underf lowing, the quantisation factor used in the encoder 
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can be decreased causing more data to be produced to fill the decoder buffer. If the decoder buffer is in danger of over- 
flowing, the quantisation factor used in the encoder can be increased, reducing the amount of data generated and sent 
to the decoder thereby allowing the decoder buffer to empty. This type of system, where feecfoack from the decoder 
buffer model is used to control the quantisation factor used by the encoder, is known as a closed-loop system. 

5 [0012] In the situation where a number of standard encoders are used to encode segments of a HDTV picture, the 
situation may become more complicated. Reliable decoder buffer management must still be performed, however the 
problem is compounded by having to deal with many separate encoders. The standard closed-loop approach could be 
used, wherein each encoder alters the quantisation factor to control the occupancy of a decoder buffer. It is most desir- 
able to use predominantly standard equipment, however, standard decoders and equipment are not designed to be 

w used in such a configuration. 

[OOI 3] Accordingly, one aspect of the present invention is to provide a method and apparatus for managing a decoder 
buffer for use in substantially standard encoder systems. 

[0014] According to a first aspect of the present invention there is provided a method of transmitting a digital signal 
comprising a succession of encoded quantities of information each quantity being of a variable size and representing 

is information intended to be received by a decoder incorporating a decoder buffer, wherein each quantity of information 
has an associated decode time indicating the time at which the quantity should be removed from the decoder buffer by 
the decoder, the method comprising the steps of: transmitting the digital signal as a bitstream; biasing the bitrate of the 
bitstream to cause the decoder buffer to overflow; predicting from the decode times and size of the quantities of infor- 
mation an overflow of the decoder buffer; and adding stuffing bits to at least one quantity of information to be received 

20 by the decoder buffer prior to any predicted overflow to prevent the decoder buffer from overflowing. 

[001 5] According to a second aspect of the present invention there is provided a method for transmitting a television 
signal comprising a succession of encoded frames each representing picture information intended to be received by a 
decoder incorporating a decoder buffer, wherein each frame has an associated decode time indicating the time at which 
the frame should be removed from the decoder buffer the method comprising the steps of: generating a plurality of indi- 

25 vidual bitstreams each of which represents respective picture elements in a composite array of such picture elements; 
combining the individual bitstreams to form the said television signal such that each frame is represented by a variable 
number of encoder frame bits; initialising the transmission of the television signal in relation to the frame decode time 
to achieve at least a predetermined initial level of occupancy of the decoder buffer; adding stuffing bits to the encoded 
frame bits to limit the variations in decoder buffer occupancy caused by variations between frames in the number of 

30 frame bits; and biasing the transmission rate of the television signal to cause the buffer to overflow. 

[001 6] According to a third aspect of the present invention there is provided apparatus for transmitting a digital signal 
comprising a succession of encoded quantities of information each quantity being of a variable size and representing 
information intended to be received by a decoder incorporating a decoder buffer, wherein each quantity of information 
has an associated decoder time indicating the time at which the quantity should be removed from the decoder buffer by 

35 the decoder, the apparatus comprising: a transmitter for transmitting the digital signal as a bitstream at a bitrate to 
cause the decoder buffer to overflow; a first controller for predicting from the decode times and size of the quantities of 
information an overflow of the decoder buffer; and a second controller for adding stuffing bits to at least one quantity of 
information to be received by the decoder buffer prior to the predicted overflow to prevent the decoder buffer from over- 
flowing. 

40 [001 7] According to a fourth aspect of the present invention there is provided apparatus for transmitting a television 
signal comprising a succession of encoded frames each representing picture information intended to be received by a 
decoder incorporating a decoder buffer, wherein each frame has an associated decode time indicating the time at which 
the frame should be removed from the decoder buffer the apparatus comprising: a generator for generating a plurality 
of individual bitstreams which represent respective picture elements in a composite array of such picture elements; a 

45 first controller for combining the individual bitstreams to form the said television signal such that each frame is repre- 
sented by a variable number of encoder frame bits; a second controller for initialising the transmission of the television 
signal in relation to the frame decode times to achieve at least a predetermined initial level of occupancy of the decoder 
buffer; a third controller for adding stuffing bits to the encoded frame bits to limit the variations in decoder buffer occu- 
pancy caused by variations between frames in the number of frame bits; and a fourth controller for varying to transmis- 

so sion rate to cause the buffer to overflow. 

[0018] The invention will now be described by way of example, with reference to the accompanying drawings, in 
which: 

Figure 1 is a diagram showing an overview of the present invention; 
55 Figure 2 is a diagram showing a high definition television image divided into a number of elements; 

Figure 3 is diagram showing a number of examples of how a typical decoder buffer may vary over time; 

Figure 4 is a~diagram showing an overview of a first embodiment of the present invention; 

Figure 5 is a diagram showing the relationship between buffer occupancy, bitrate and transmission start time; and 
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Figure 6 is a diagram showing an overview of a second embodiment of the present invention. 

[0019] Referring now to Figure 1 , there is shown a diagram showing an overview of the present invention. Figure 2 
shows a large picture 200 which is divided up into a number of smaller segments 201 by a splitter 107. Each of the seg- 

5 ments are fed to separate standard encoders 101 to produce a number of individual digitally encoded bitstreams 105. 
These individual bitstreams are then input to a multiplexer and combiner 102 where each of the bitstreams is combined 
and processed to produce a single HDTV bitstream 103. This HDTV bitstream is then transmitted via a suitable trans- 
mission channel to a suitable decoder 104 having a decoder buffer. Further details of the above-described system can 
be found in our co-pending United Kingdom patent application, GB 9807203.6. 

io [0020] Turning now to Figure 3, there are shown a number of examples of how the occupancy of a video decoder 
buffer may typically change over time. In each of the examples, a constant input bitrate is assumed, although the same 
principles apply equally to variable bitrates. 

[0021] In a first example, Figure 3a shows how the occupancy of a video decoder buffer may typically change over 

time. Frames having a decoder time-stamp (DTS) are removed from the video decoder buffer at a constant rate, as 
15 determined by the frame rate. As a frame is removed from the decoder buffer the buffer occupancy reduces by the size 

of the removed frame. In this example, the decoder buffer neither underflows nor overflows. 

[0022] Figure 3b shows another example of how the occupancy of a decoder buffer may typically change over time. 

It can be seen that although frames are being removed from the decoder buffer, the overall occupancy of the decoder 

buffer is increasing. However, at 301 the buffer has overflowed, since the frame data rate into the decoder buffer is 
20 greater than the rate in which frame data is removed from the decoder buffer. Buffer overflow causes frame data to be 

lost and the decoding process will breakdown. 

[0023] Figure 3c shows yet another example of how the occupancy of a video decoder buffer may typically change 
over time. In this example, the frame data rate into the decoder buffer is lower than the rate at which frame data is being 
removed. At 302, the decoder has attempted to remove a frame from the decoder, but the frame was not entirely 

25 present in the video decoder buffer at that time. Buffer underflow may also cause the decoding process to breakdown. 
[0024] Successful management of the decoder buffer is therefore of vital importance to maintain the integrity of the 
decoding process. In the case of the present invention the amount of data produced cannot be altered, but the decoder 
buffer must still be successfully managed so that the occurrence of decoder buffer overflow or underflow is minimised. 
[0025] In a situation where multiple independent standard encoders are used to produce higher definition television 

30 pictures, as already previously described, problems may arise especially where the combined capacity of each of the 
independent encoders buffers is greater than the permissible video buffer capacity of an associated HDTV decoder 
buffer. Such a problem may be encountered when using 6 standard encoders to produce HDTV at the highest resolu- 
tions (e.g. 1080 x 1920 pixels at 30 frames per second). The problem occurs since it is possible for a single video frame 
to entirely fill a standard video decoder buffer. If each of the six separate encoders were to simultaneously produce such 

35 a frame, the decoder video buffer would overflow (assuming the combined capacity of each of the separate encoder 
buffers was greater than the HDTV decoder buffer). 

[0026] A standard decoder buffer, as defined by the Motion Pictures Expert Group (MPEG) MP@ML (Main Profile at 
Main Level) has a capacity of 1.8Mbrts. A decoder buffer at MP@HL (Main Profile at High Level) has a capacity of 
9.8Mbits, and a decoder buffer as defined by the American Television Standards Committee (ATSC) has a capacity of 
40 8Mbits. For both of these high definition standards the total capacity of six standard decoder buffers is greater than 
capacity of a high definition buffer. 

[0027] One way to overcome this problem would be to increase the size of the HDTV decoder buffer, but the con- 
straints applied by the various standards groups prevent this. It is, however, necessary to ensure that the combined 
capacity of each of the separate encoder buffers is no greater than the capacity of the HDTV decoder buffer. In this 
-45 case, the effective size of each of the separate encoder buffers must be reduced accordingly, to ensure that overflow of 
the combined decoder buffer cannot occur. 

[0028] In Figure 4 a first embodiment of the present invention is shown in which the occurrence of buffer overflow and 
underflow is minimised through an appropriate buffer initialisation. Initialisation of the decoder buffer involves ensuring 
that the occupancy of the decoder buffer is at a predetermined level for an initial frame. If the initial buffer occupancy 

so level chosen is too low, then the decoder buffer may underflow. If the initial buffer occupancy level chosen is too high, 
then the decoder buffer may overf tow. Once the initial buffer occupancy level and DTS is chosen the encoder must cal- 
culate when to start outputting data so that the above described criteria are met in the decoder buffer. 
[0029] Referring now to Figure 4, there is shown an overview of a first embodiment of the present invention. A number 
of standard encoders 101 are connected to a modified multiplexer and combiner 602, to produce a HDTV bitstream 

55 603. A decoder buffer model controller 600 maintains a model of the occupancy of a decoder buffer for each of the indi- 
vidual encoders 101. 

[0030]" Prior to transmission of the HDTV signal a decoder buffer initialisation controller 601 calculates the start-time 
of the transmission to ensure that the HDTV decoder buffer is at a particular predetermined occupancy at the time when 
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the first frame of a sequence is removed from the decoder buffer. 

[0031] The accurate HDTV decoder buffer occupancy for initialisation is the combined occupancy of each of the indi- 
vidual standard decoder occupancies for the chosen start frame, ft is also important to ensure that the total combined 
capacity of each of the individual decoder buffers does not exceed the capacity of the HDTV decoder buffer. 
5 [0032] Once the required decoder buffer occupancy for initialisation is known the transmission start time can be cal- 
culated. 

[0033] Figure 5 shows how the decoder buffer initialisation controller 601 calculates the transmission start time. The 
desired buffer occupancy 500 is that of the combined occupancies of each of the individual buffer models as determined 
by the decoder buffer model controller 600. The time by which that occupancy level should be reached is the time 501, 
w or the DTS time of the initial frame of the chosen sequence. For ease of explanation a constant bit rate 503 is assumed, 
although the same principles apply also for variable bitrates. The start time can thus be expressed as: 

StartTime = DTS- ° CCUpanCy 
is Bitrate 

Equation 1 



20 

[0034] DTS is known, being the first frame of the transmission. Occupancy is the predetermined occupancy, deter- 
mined by the decoder buffer model controller 600. 

[0035] Once initialisation has taken place, the decoder buffer should ideally never underflow or overflow, assuming 
that the encoders remain within their predefined encoding limits. 
25 [0036] Where it is not desirable to monitor the occupancies of each of the separate encoder buffers, a somewhat sim- 
pler approach can be taken, as illustrated in Figure 6. Figure 6 is a diagram showing an overview of a second embodi- 
ment of the present invention. 

[0037] A number of standard encoders 1 01 are connected to a modified multiplexer and combiner 703. The individual 
bitstreams are combined and multiplexed together by the multiplexer and combiner module 705 to produce a HDTV sig- 

30 nal. A start time controller 700 calculates when to start transmitting the HDTV signal, such that a decoder buffer in a 
decoder is suitably initialised. In this embodiment, the initial decoder buffer occupancy level chosen for initialisation is 
not precisely calculated, unlike in the first embodiment, but a predetermined occupancy level is chosen. The chosen 
occupancy level needs to be sufficiently high so that immediate buffer underflow does not occur, but a small operating 
margin at the top of the buffer occupancy is also preferred. In the present embodiment a level of around 70 to 95% buffer 

35 capacity is chosen. 

[0038] Once the required initialisation buffer occupancy is chosen, the start time can be calculated as per Equation 1 . 
[0039] Since the initialisation buffer occupancy is not accurately chosen, compared to the method of the first embod- 
iment, the decoder buffer is still in danger of underf lowing and/or overflowing. To reduce the possibility of buffer under- 
flow, the HDTV bitstream signal 704 is transmitted to the decoder at a rate greater than the average rate at which data 
40 is removed from the decoder buffer. This will cause the decoder buffer to tend to overf taw, thus minimising the possibility 
of buffer underflow. 

[0040] To minimise the possibility of decoder buffer overflow a buffer overflow protection controller 702 is used, in con- 
junction with a HDTV decoder buffer model 701 . 

[0041] The HDTV decoder buffer model 701 models the occupancy of a HDTV decoder buffer in accordance with the 
45 HDTV signal. Once all the frame data for a frame has been transmitted to the decoder, the buffer overflow protection 
controller 702 predicts whether the decoder buffer will overflow before the next frame to be transmitted to the decoder 
is removed from the decoder buffer. If decoder buffer overflow is predicted to occur, the buffer overflow protection con- 
troller 702 adds stuffing to the end of the current frame to create enough space in the decoder buffer for the next frame 
to be transmitted to the buffer. Referring back to Figure 3, the function of the buffer overflow protection controller is 
so shown in more detail. 

[0042] Figure 3b shows buffer overflow occurring at 301 . Figure 3d shows the same buffer occupancy graph as Figure 
3b but with the buffer overflow protection in place. The buffer overflow protection controller 702 calculates that the buffer 
will overflow at 301 , at the time DTS3, and also calculates the amount by which the buffer will overf tow. To prevent the 
buffer from overflowing, stuffing data is added to the frame to be removed from the decoder buffer at time DTS2 (Figure 
55 3d). This has the effect of freeing up sufficient space in the decoder buffer such that buffer no longer overflows at time 
DTS3. 

[0043]- With reference to Figure 5 the method by which "decoder buffer overflow is calculated is explained. 
[0044] The occupancy of the decoder buffer can be calculated: 
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Occupancy = (DTS - StartTime) * Bitrate 

[0045] Since the size of the frame is known, the amount of free space in the decoder buffer after the frame has been 
removed is given by: 

5 

FreeSpace - BufferSize - {Occupancy - FrameSize) 

[0046] The frame rate is also known which allows a prediction to be made as to whether the decoder buffer will over- 
flow before the next frame is removed from the decoder buffer. The amount of data input to a decoder buffer in a frame 
io period is thus: 

TotalData = FramePeriod * Bitrate 
[0047] The decoder buffer will therefore overflow if: 

15 

FreeSpace < TotalData 

[0048] If the decoder buffer will overflow, stuffing can be added to the current frame, such that when that frame is 
removed from the decoder buffer enough space is made available in the decoder buffer to avoid overflow before the next 
20 frame is removed from the buffer. 

[0049] The amount of stuffing to be added is thus: 

Stuffing = {FramePeriod * Bitrate) - FreeSpace 

25 [0050] The present invention is of particular application in the field of encoding high definition television signals, 
although it would be appreciated by anyone skilled in the art that it could also be applied to other applications where a 
large amount of data is encoded using a number of standard encoders. Such applications could include audio encoding 
and data processing. 

30 Claims 

1 . A method of transmitting a digital signal comprising a succession of encoded quantities of information each quantity 
being of a variable size and representing information intended to be received by a decoder incorporating a decoder 
buffer, wherein each quantity of information has an associated decode time indicating the time at which the quantity 

35 should be removed from the decoder buffer by the decoder, the method comprising the steps of: 

transmitting the digital signal as a bitstream; 

biasing the bitrate of the bitstream to cause the decoder buffer to overflow; 

predicting from the decode times and size of the quantities of information an overflow of the decoder buffer; 
40 and 

adding stuffing bits to at least one quantity of information to be received by the decoder buffer prior to any pre- 
dicted overflow to prevent the decoder buffer from overflowing. 

2. A method for transmitting a television signal comprising a succession of encoded frames each representing picture 
45 information intended to be received by a decoder incorporating a decoder buffer, wherein each frame has an asso- 
ciated decode time indicating the time at which the frame should be removed from the decoder buffer the method 
comprising the steps of: 

generating a plurality of individual bitstreams each of which represents respective picture elements in a corn- 
so posite array of such picture elements; 

combining the individual bitstreams to form the said television signal such that each frame is represented by a 
variable number of encoder frame bits; 

initialising the transmission of the television signal in relation to the frame decode time to achieve at least a pre- 
determined initial level of occupancy of the decoder buffer; 
55 adding stuffing bits to the encoded frame bits to limit the variations in decoder buffer occupancy caused by var- 

iations between frames in the number of frame bits; and 

biasing the transmission rate of the television signal to cause the buffer to overflow. 
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3. The method of claim 1, wherein the step of adding stuffing bits consists of adding stuffing bits to one quantity of 
information only. 

4. The method of claim 1 or 3, wherein the step of predicting an overflow of the buffer further comprises calculating / 
5 the amount by which an overflow wiiroccur to determine the number of stuffing bits. 

5. The method of claim 2, further comprising limiting the maximum size of the plurality of encoded picture elements 
to a predetermined size. 

10 6. The method of claim 2 or 5, wherein the predetermined size is the capacity of the decoder buffer, 

7, The method of claim 2, 5 or 6, wherein the step of initialising to a predetermined initial level of occupancy initialises 
the decoder buffer to an occupancy of between 70 and 95% of the buffer capacity. 

15 8. Apparatus for transmitting a digital signal comprising a succession of encoded quantities of information each quan- 
tity being of a variable size and representing information intended to be received by a decoder incorporating a 
decoder buffer, wherein each quantity of information has an associated decoder time indicating the time at which 
the quantity should be removed from the decoder buffer by the decoder, the apparatus comprising: 

20 a transmitter for transmitting the digital signal as a bitstream at a bitrate to cause the decoder buffer to overflow; 

a first controller for predicting from the decode times and size of the quantities of information an overflow of the 
decoder buffer; and 

a second controller for adding stuffing bits to at least one quantity of information to be received by the decoder 
buffer prior to the predicted overflow to prevent the decoder buffer from overflowing. 

25 

9. Apparatus for transmitting a television signal comprising a succession of encoded frames each representing picture 
information intended to be received by a decoder incorporating a decoder buffer, wherein each frame has an asso- 
ciated decode time indicating the time at which the frame should be removed from the decoder buffer the apparatus 
comprising: 

30 • 

a generator for generating a plurality of individual bitstreams which represent respective picture elements in a 
composite array of such picture elements; 

a first controller for combining the individual bitstreams to form the said television signal such that each frame 
is represented by a variable number of encoder frame bits; 
35 a second controller for initialising the transmission of the television signal in relation to the frame decode times 

to achieve at least a predetermined initial level of occupancy of the decoder buffer; 

a third controller for adding stuffing bits to the encoded frame bits to limit the variations in decoder buffer occu- 
pancy caused by variations between frames in the number of frame bits; and 
a fourth controller for varying to transmission rate to cause the buffer to overflow. 

40 

10. The Apparatus of claim 8, wherein the second controller is adapted to add stuffing bits to one quantity of informa- 
tion only. 

11. The Apparatus of claim 8 or 10, wherein the first controller further comprises a calculator to calculate the amount 
45 by which an overflow will occur to determine the number of stuffing bits. 
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